Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Elasticsearch-DSL code #2736

Merged
merged 11 commits into from
Jan 17, 2025
Merged

Conversation

miguelgrinberg
Copy link
Contributor

@miguelgrinberg miguelgrinberg commented Jan 10, 2025

This change adds the Elasticsearch-DSL code. Note that the documentation will be merged in a separate PR.

  • The imported source tree has the same structure as in the elasticsearch_dsl package, except that its root is now elasticsearch.dsl. Users can replace the _ with a . in all imports.
  • I had to make minor changes to some of the DSL's test fixtures, because this client uses function scope for the client fixtures and DSL used session scope. I have switched all the DSL fixtures to function scope.
  • I had to make the DSL client fixtures work with https:// Elasticsearch endpoints, since that is what we are using in integration test runs.
  • The unasync of the DSL classes remains in a separate run-unasync-dsl.py script. We can consider merging the two.
  • The generator.py script that generates the query, aggregation and response source files was moved to this project as generator-dsl.py, along with its Jinja templates. We can consider moving this to the Python client generator project.

Copy link

A documentation preview will be available soon.

Request a new doc build by commenting
  • Rebuild this PR: run docs-build
  • Rebuild this PR and all Elastic docs: run docs-build rebuild

run docs-build is much faster than run docs-build rebuild. A rebuild should only be needed in rare situations.

If your PR continues to fail for an unknown reason, the doc build pipeline may be broken. Elastic employees can check the pipeline status here.

@miguelgrinberg miguelgrinberg force-pushed the dsl-merge branch 3 times, most recently from 4ec2881 to 7b1af33 Compare January 10, 2025 16:21
@miguelgrinberg miguelgrinberg force-pushed the dsl-merge branch 11 times, most recently from 1d951c6 to 9cab9c2 Compare January 15, 2025 12:51
@miguelgrinberg miguelgrinberg marked this pull request as ready for review January 15, 2025 16:44
@miguelgrinberg
Copy link
Contributor Author

miguelgrinberg commented Jan 15, 2025

@pquentin The builds are currently not generating code coverage data, and I haven't been able to figure out why. And I get coverage data when I run the tests locally. I'll continue looking for this, but if you have any ideas why this is happening let me know. Other than this the PR is ready for a review.

Copy link
Member

@pquentin pquentin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The unasync of the DSL classes remains in a separate run-unasync-dsl.py script. We can consider merging the two.
  • The generator.py script that generates the query, aggregation and response source files was moved to this project as generator-dsl.py, along with its Jinja templates. We can consider moving this to the Python client generator project.

Yes, please, as a follow-up.

@pquentin The builds are currently not generating code coverage data, and I haven't been able to figure out why. And I get coverage data when I run the tests locally. I'll continue looking for this, but if you have any ideas why this is happening let me know. Other than this the PR is ready for a review.

I haven't looked in detail yet, but the current build has 13% coverage, and we don't use coverage for anything meaningful today. However it would be nice to fix this issue as I'm seeing 0% coverage here. (In the future, we can exclude generated code to make coverage more useful.)

elasticsearch/dsl/__init__.py Outdated Show resolved Hide resolved
elasticsearch/dsl/py.typed Outdated Show resolved Hide resolved
noxfile.py Outdated Show resolved Hide resolved
noxfile.py Show resolved Hide resolved
test_elasticsearch/test_dsl/__init__.py Outdated Show resolved Hide resolved
elasticsearch/dsl/document_base.py Outdated Show resolved Hide resolved
examples/dsl/async/sparse_vectors.py Outdated Show resolved Hide resolved
@miguelgrinberg
Copy link
Contributor Author

@pquentin The last commit addresses your feedback. One thing I want to direct your attention to, is the handling of the user agent, which I only noticed now. The connection class used to add an elasticsearch-dsl-py/$VERSION user agent. I changed that to use elasticsearch-py/$VERSION, but I can't decide if this is a good idea or we should better leave it with the DSL user agent in case we want to see how many people use the DSL classes going forward. Let me know what your thoughts are on this.

Copy link
Member

@pquentin pquentin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Sorry, I did not realize my mypy suggestion would bring so much changes.

elasticsearch/dsl/connections.py Outdated Show resolved Hide resolved
elasticsearch/dsl/connections.py Outdated Show resolved Hide resolved
Copy link
Member

@pquentin pquentin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM. 🚀

@miguelgrinberg miguelgrinberg merged commit 10ded22 into elastic:main Jan 17, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants